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OBJECT: To provide a moni- 
toring camera system in 
which the image from the 
monitoring camera for auto- 
matically tracking a moving 
body is subject to little shake, 
and in which there is little 
load on the universal head. 

CONSTITUTION: Comprises a 
moving body detection 
camera ^11 capable of 
capturing images of a moving 
body over a wide image 
capture area; a monitoring 
camera 21 for acquiring an 
enlarged image of a moving 
body; a first and a second attitude control means 12 and 22 for controlling the attitudes of 
moving body detection camera 11 and monitoring camera 21 respectively; a first and a second 
camera angle of view storage means 31 and 32 serving to give first and second attitude control 
means 12 and 22 instructions by calling [1]* a plurality of angle of view information items that 
have been stored in advance; and image processing unit 40 for extracting features of a moving 
body from the image from moving body detection camera 11, and for selecting appropriate 
information on the basis of the plurality of angle of view information items that have been 
stored in the first and the second camera angle of view storage means 31 and 32. 
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Claims 

1. A monitoring camera system characterised in tinat it comprises: 

a moving body detection camera capable of capturing images of a moving body 
over a wide image capture area; 

a monitoring camera for acquiring an enlarged image of a moving body; 

a first and a second attitude control means for controlling the attitudes of said 
moving body detection camera and said monitoring camera respectively; 

a first and a second camera angle of view storage means for giving said first and 
second attitude control means instructions by calling a plurality of angle of view 
information items that have been stored in advance; 

and an image processing unit for extracting features of a moving body from the 
image from said moving body detection camera, and for selecting appropriate 
information on the basis of the plurality of angle of view information items that have 
been stored in said first and second camera angle of view storage means. 



* Numbers in square brackets refer to Translator's Notes appended to the translation. 
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2. The monitoring camera system according to Claim 1, which captures a moving 
body within the Image capture area of said monitoring camera by means of discreet 
control of said second attitude control means, this control being effected by switching 
camera angle of view Information comprising horizontal angle of view Information, 

5 vertical angle of view Information and increase/decrease angle of view information, 
these Items of information having been established in advance. 

3. The monitoring camera system according to Claim 1, which, provided that the 
moving body does not go outside an area defined as being a set distance from the 

10 centre of the screen of said monitoring camera, does not cause said monitoring 
camera to swivel. 

4. The monitoring camera system according to Claim 1, which captures a moving 
body in the Image capture area of said monitoring camera while decreasing 

15 Inappropriate switching of angle of view information of said monitoring camera, by 
setting adjacent camera angle of view Information In such manner that it overlaps 
with the image capture area of the camera. 

5. The monitoring camera system according to Claim 1, which captures a moving 
20 body In the image capture area of said moving body detection camera, by causing 

said moving body detection camera to swivel in the direction of movement of the 
moving body before the moving body leaves the image capture area of said moving 
body detection camera. 

25 6. The monitoring camera system according to Claim 1, wherein said image 
processing unit comprises: 

a moving body extraction means for extracting a moving body from the image 
from said moving body detection camera; 

a feature and position extraction means for extracting the features and positions 
30 of an individual moving body on the basis of the extraction result of said moving body 
extraction means; 

a feature and position information storage means for storing past feature and 
position information; 

a target body tracking means for deciding, from the features of the moving body, 
35 whether or not it is a target body, and for correlating it with a past target body that 
has been stored in said feature and position Information storage means; and 

a camera angle of view specifying means for Indicating to said camera angle of 
view storage means a camera angle of view corresponding to the tracking result. 

Detailed Description of the Invention 
40 Technical field of the invention 

(1) The present Invention relates to monitoring camera systems for tracking, 
capturing images of, and monitoring a moving body.* 



* Numbers in round brackets at tiie beginning of paragraphs correspond to the paragraph numbering in 
the Japanese patent document. 
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Prior art 

(2) A conventional monitoring camera system generally has a moving body detection 
camera and a decision camera [2], detects a moving body by means of the moving 
body detection camera, and causes the universal head of the decision camera to 

5 swivel in such manner that the moving body in question is located in the centre of the 
decision camera screen. An example of such a monitoring camera system is set forth 
in Japanese Unexamined Patent Publication No. 07-037100. 

(3) The monitoring camera system set forth in this publication comprises a moving 
body detection camera, a decision camera, a decision processing unit, and an 

10 information processing unit. Using these two cameras, the system reliably detects — 
over a wide image capture area with no blind spots — a moving body and captures 
images of this moving body in a condition required for a decision. The moving body 
detection camera monitors a 360° area by means of an approximately conical 
reflecting mirror arranged above it, and outputs information relating to the position 

15 and area of the moving body. The information processing unit transforms coordinates 
on the reflecting mirror to coordinates on the monitoring area plane. The decision 
processing unit decides on the swivel angle and the angle of inclination or declination 
on the basis of the position information, and also decides on magnification on the 
basis of the area information, in such manner that the moving body is located in the 

20 centre of the screen of the decision camera. The decision camera captures the moving 
body by swivelling on the basis of the output of the decision processing unit. 

(4) Another example of a monitoring camera system is set forth in Japanese 
Unexamined Patent Publication No. 59-221191. The monitoring camera system 
described in this publication comprises a television camera, a video memory, a control 

25 circuit, a common memory, an output image memory, an output circuit, an output 
device, a change extraction circuit, an outline extraction circuit, a centre extraction 
circuit and a tracking circuit, and causes the television camera to move so as to 
match the movement of a moving body that has entered the angle of view of the 
television camera. The video memory successively updates and stores images that 

30 have been input by the television camera. The control circuit controls each of the 
other circuits and also controls the flow of information. The common memory is 
accessed by each of the other circuits. The output image memory stores image 
information to be output to the output device. The output circuit converts information 
in the output image memory to a video signal and displays images on the output 

35 device. The change extraction circuit finds the difference between the common 
memory and the output image memory, and extracts the change component between 
the current and previous input images. The outline extraction circuit extracts the 
outline of a moving body that is present when there has been a change. The centre 
extraction circuit extracts the centre or centre of gravity of the moving body on the 

40 basis of the outline information obtained by the outline extraction circuit. The tracking 
circuit causes the centre of the lens of the video camera to move to the centre of the 
moving body in response to a change in that centre. 

Problems which the invention will solve 

(5) The following sorts of problems are encountered in conventional monitoring 
45 camera systems including the examples described above. 
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(6) The first problem is that when a moving body stops, it takes time for shaking of 
the camera image to settle down. This is because the universal head is controlled in 
such manner that the centre of gravity of the moving body is always kept positioned 
in the centre of the screen. 

5 (7) The second problem is that the camera image shakes while the moving body is 
being tracked. This is because the universal head is controlled in such manner that 
the extracted centre of gravity of the moving body is always kept positioned in the 
centre of the screen despite the fact that because the extracted centre of gravity of 
the moving body is not always absolutely In agreement with the centre of gravity of 

10 the actual moving body, the locus of the extracted centre of gravity does not describe 
a smooth curve. 

(8) The third problem is that the computation required for controlling the universal 
head is complicated. This is because in order to control the universal head so that the 
centre of gravity of the moving body is always kept positioned in the centre of the 

15 screen, not only the movement of the moving body but also the characteristics of the 
universal head have to be taken into consideration. 

(9) The fourth problem is that the universal head has a short lifetime. The reason for 
this is that the universal head is subjected to loading in the course of performing the 
precise control required to ensure that the centre of gravity of the moving body is 

20 always kept positioned in the centre of the screen. 

(10) A task for the present invention is to provide a monitoring camera system 
capable of tracking a moving body with little image shake. 

(11) A further task for the present invention is to provide a monitoring camera system 
with simplified processing. 

25 (12) Yet another task for the present invention is to provide a monitoring camera 
system in which control commands are only infrequently given to the universal head, 
and in which there is little component wear. 

Means for solving problems 

(13) The present invention provides a monitoring camera system characterised in that 
30 it comprises: a moving body detection camera capable of capturing images of a 

moving body over a wide image capture area; a monitoring camera for acquiring an 
enlarged image of a moving body; a first and a second attitude control means for 
controlling the attitudes of the moving body detection camera and the monitoring 
camera respectively; a first and a second camera angle of view storage means for 

35 giving the first and second attitude control means instructions by calling a plurality of 
angle of view information items that have been stored in advance; and an image 
processing unit for extracting features of a moving body from the image from the 
moving body detection camera, and for selecting appropriate information on the basis 
of the plurality of angle of view information items that have been stored in the first 

40 and second camera angle of view storage means. 

(14) The present invention also provides a monitoring camera system which captures 
a moving body within the image capture area of the monitoring camera by means of 
discreet control of the second attitude control means, this control being effected by 
switching camera angle of view information comprising horizontal angle of view 
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information, vertical angle of view information and increase/decrease angle of view 
information, these items of information having been established In advance. 

(15) The present invention further provides a monitoring camera system which, 
provided that the moving body does not go outside an area defined as being a set 

5 distance from the centre of the screen of the monitoring camera, does not cause the 
monitoring camera to swivel. 

(16) The present invention also provides a monitoring camera system which captures 
a moving body in the image capture area of the monitoring camera, while decreasing 
inappropriate switching of angle of view information of the monitoring camera, by 

10 setting adjacent camera angle of view Information in such manner that it overlaps 
with the Image capture area of the camera. 

(17) The present Invention still further provides a monitoring camera system which 
captures a moving body in the image capture area of the moving body detection 
camera, by causing this moving body detection camera to swivel In the direction of 

15 movement of the moving body before the moving body leaves the Image capture area 
of the moving body detection camera. 

(18) The present invention also provides a monitoring camera system wherein the 
image processing unit comprises: a moving body extraction means for extracting a 
moving body from the image from the moving body detection camera; a feature and 

20 position extraction means for extracting the features and positions of an individual 
moving body on the basis of the extraction result of the moving body extraction 
means; a feature and position information storage means for storing past feature and 
position information; a target body tracking means for deciding, from the features of 
the moving body, whether or not it is a target body, and for correlating it with a past 

25 target body that has been stored in the feature and position information storage 
means; and a camera angle of view specifying means for indicating to the camera 
angle of view storage means a camera angle of view corresponding to the tracking 
result. 

Working of the invention 

30 (19) When moving bodies enter an image capture area of the moving body detection 
camera, the moving body extraction means extracts the moving bodies, the feature 
and position extraction means obtains the features and positions of the moving 
bodies, and the target body tracking means, while referring to information in the 
feature and position information storage means, decides which of the moving bodies 

35 to track as a target body, and obtains its position. On the basis of the position of the 
target body being tracked, the camera angle of view specifying means selects the 
most suitable of the items of information that have been stored In the camera angle 
of view storage means. The camera angle of view storage means controls the attitude 
control means on the basis of the selected camera angle of view information. 

40 Consequently, the present invention Is capable of capturing a moving body with the 
monitoring camera by means of the minimum amount of attitude control. 

Mode of embodying the invention 

(20) A mode of embodying the monitoring camera system of the present invention 
will now be described with reference to the drawings. 
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(21) FIG. 1 is a block diagram showing the constitution of this mode of embodying the 
monitoring camera system of the present Invention. Referring to FIG. 1, it will be seen 
that this monitoring camera system comprises: moving body detection camera 11; 
attitude control means 12 serving as the first attitude control means and being a 

5 universal head or the like for determining the attitude of moving body detection 
camera 11; monitoring camera 21; attitude control means 22 serving as the second 
attitude control means and being a universal head or the like for determining the 
attitude of monitoring camera 21; image output device 23 such as a display for 
displaying the image from monitoring camera 21; camera angle of view storage 

10 means 31 serving as the first camera angle of view storage means and being a preset 
controller or the like for controlling attitude control means 12; camera angle of view 
storage means 32 serving as the second camera angle of view storage means and 
being a preset controller or the like for controlling attitude control means 22; and 
Image processing unit 40. 

15 (22) Image processing unit 40 comprises moving body extraction means 41, feature 
and position extraction means 42, feature and position information storage means 43, 
target body tracking means 44 and camera angle of view specifying means 45. 

(23) Camera angle of view storage means 31 and 32 store, in advance, a plurality of 
angle of view information items, namely, horizontal angle of view (pan), vertical angle 

20 of view (tilt) and increase/decrease angle of view (zoom), in such manner that no 
part of the monitoring area is left uncovered. [3] 

(24) Camera angle of view specifying means 45 has a table in which the angle of view 
information stored in camera angle of view storage means 31 and 32 is linked to 
positions within the monitoring area. 

25 (25) When a moving body enters the image capture area of moving body detection 
camera 11, moving body extraction means 41 of image processing unit 40 extracts 
the region within the screen corresponding to the moving body. Feature and position 
extraction means 42 obtains the features and position for each moving body region. 
Target body tracking means 44 compares the information relating to each moving 

30 body [4] with past moving body information that has been stored In feature and 
position information storage means 43, uses the closest match as the current target 
body, and updates the information in feature and position information storage means 
43. Camera angle of view specifying means 45 selects camera angles of view such 
that the position of the target body obtained by target body tracking means 44 is 

35 located within the Image capture areas and, if and only if these differ from the current 
camera angles of view, outputs, to camera angle of view storage means 31 and 32, 
the table number or numbers corresponding to the selected camera angle or angles of 
view. [5] Camera angle of view storage means 31 and 32 call the camera angle of 
view information from the input table number or numbers and on this basis control 

40 attitude control means 12 and 22. 

(26) FIG. 2 and FIG. 3 are flowcharts serving to clarify the operation of this system. 
FIG. 4, FIG. 5, FIG. 6, FIG. 7, FIG. 8, FIG. 9 and FIG. 10 are drawings also serving to 
clarify the operation of this system, and respectively give examples of: an input 
image, an extracted moving body, image processing blocks and the monitoring 
45 camera angle of view (for an example where the image capture area of the moving 
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body detection camera has been divided into 58 5 blocks) [6], monitoring camera 
swivel, outermost Image processing blocks, moving body detection camera swivel 
directions, and the overlap and swivel applied to the angle of view of the moving body 
detection camera. 

5 (27) The operation of this system will now be described with reference to FIGS. 1 to 
10. 

(28) At step SI, if there is an input image (of the sort shown in FIG. 4) from moving 
body detection camera 11 of FIG, 1, moving body extraction means 41 extracts and 
binarizes the moving body (see FIG. 5). At step S2, it is decided whether or not there 

10 is a moving body. If there is no moving body, processing returns to step SI. 

(29) At step S3, unique numbers are assigned to regions on the extracted image 
(such as that depicted in FIG. 5). This processing step is termed "labelling". 

(30) At step S4, feature and position extraction means 42 firstly obtains, from the 
result of step S3, the feature vectors for all the regions, decides whether or not there 

15 is a body to be tracked (for example a person), and deletes data relating to other 
bodies (e.g., the swaying of trees). Next, it obtains representative coordinates 
(centre, lower edge, upper edge, etc.) of the remaining data. 

(31) At steps S5 to S7, target body tracking means 44 searches for data where there 
is agreement between (i) all the current feature vector data and (ii) the latest data in 

20 the sequence of past feature vector data of feature and position storage means 43, 
and determines the position of the target body to be tracked. This processing is 
performed by going back to past data until such matching data are found. 

(32) At step S8, when matching data are found, the oldest data in feature and 
position storage means 43 are deleted and the storage means is updated with the 

25 matching data. The screen [7] of moving body detection camera 11 is split into a 
number of blocks in correspondence with the angle of view information items that 
have been set in camera angle of view storage means 31, and camera angle of view 
storage means 32 has preset angles of view such that monitoring camera 21 captures 
an enlarged image of each of these blocks (see FIG. 6). In camera angle of view 

30 storage means 31, angles of view that have been shifted in eight adjacent directions 
(see FIG. 9) are preset (see FIG. 10 for an example) in such manner that they 
overlap with the angle of view information items stored in camera angle of view 
storage means 32. 

(33) At step S9, camera angle of view specifying means 45 determines a new camera 
35 angle of view table number from the current angle of view of monitoring camera 21 

and from the position of the tracked target body on the image blocks (see FIG. 7). If 
the tracked target body is positioned on an outermost block (see FIG. 8), the new 
camera angle of view table number is determined in accordance with one of the swivel 
directions shown in FIG. 9, these being the swivel directions that can be applied to 
40 moving body detection camera 11. 

(34) At steps Slla and Sllb, if and only if the respective new camera angle of view 
table numbers are different from the camera angle of view table numbers in the 
immediately preceding processing cycle, these table numbers are specified for camera 
angle of view storage means 31 and 32. On the basis of this instruction by camera 
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angle of view specifying means 45, camera angle of view storage means 31 and 32 
send, to attitude control means 12 and 22 [8], the attitude information (pan, tilt and 
zoom information) that is managed by the table numbers. 

(35) At steps S12a and S12b, attitude control means 12 and 22 change the attitude 
5 by means of the attitude information that has been sent from camera angle of view 

storage means 31 and 32, and capture the tracked target body in the image capture 
field (see FIG. 7 and FIG. 9). The image is displayed on image output device 23. 

Working example 

(36) Next, a working example of a monitoring camera system according to the 
10 present invention will be described. 

(37) FIG. 11 shows the constitution of a working example of the monitoring camera 
system of the present invention. Referring to FIG. 11, it will be seen that this system 
comprises: moving body detection camera 111, its universal head 112, monitoring 
camera 121, its universal head 122, image splitter 123, display 124 for displaying the 

15 image from monitoring camera 121, VTR device 125 for recording images from the 
monitoring camera, preset controller 131 for controlling universal head 112, preset 
controller 132 for controlling universal head 122, and image processing unit 140 for 
processing the video signal from moving body detection camera 111. Note that 
depending on the functions of the universal heads, it may be feasible to incorporate 

20 the functions of preset controllers 131 and 132 in image processing unit 140. 

(38) In this system, if a moving body enters the image capture area of moving body 
detection camera 111, image processing unit 140 controls universal heads 112 and 
122 by selecting camera angle of view information stored in preset controllers 131 
and 132 and sending commands to these preset controllers 131 and 132, whereby 

25 monitoring camera 121 captures the intruding body and displays its image on display 
124. At this juncture, images of the intruder can also be recorded by sending a signal 
from image processing unit 140 to VTR device 125. 

Effect of the invention 

(39) The monitoring camera system of the present invention has the following effects. 

30 (40) Firstly, when a moving body is in the image capture area of the monitoring 
camera, camera image shake — when tracking the moving body and when the 
moving body stops — is avoided by not controlling the universal head. The reason for 
this is that because the universal head is controlled only to the minimum required 
extent, it does not react to slight movements of the moving body. 

35 (41) Secondly, by controlling the universal heads on the basis of calling camera angle 
of view information that has been stored in advance, the processing involved in 
universal head control is simplified and the loads on the universal heads are reduced. 
The reason for this is that because feedback control that takes the characteristics of 
the universal heads into account is not performed, processing is simpler and the 

40 universal heads are controlled less frequently. 
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Brief Description of the Drawings 

FIG. 1 is a block diagram showing the constitution of a mode of embodying the 
monitoring camera system of the present invention. 

FIG. 2 is a flowchart serving to clarify the operation of the monitoring camera 
5 system shown in FIG. 1. 

FIG. 3 is a flowchart serving to clarify the operation of the monitoring camera 
system shown in FIG. 1. 

FIG. 4 serves to clarify the operation of the monitoring camera system shown in 
FIG. 1 and is an example of an input image. 
10 FIG. 5 serves to clarify the operation of the monitoring camera system shown in 

FIG. 1 and is an example of an extracted moving body. 

FIG. 6 serves to clarify the operation of the monitoring camera system shown in 
FIG. 1 and is an example of image processing blocks and the monitoring camera 
angle of view (for an example where the image capture area of the moving body 
15 detection camera has been divided into 5g5 blocks). 

FIG. 7 serves to clarify the operation of the monitoring camera system shown in 
FIG. 1 and is an example of monitoring camera swivel. 

FIG. 8 serves to clarify the operation of the monitoring camera system shown in 
FIG. 1 and is an example of outermost image processing blocks. 
20 FIG. 9 serves to clarify the operation of the monitoring camera system shown in 

FIG. 1 and is an example of moving body detection camera swivel directions. 

FIG. 10 serves to clarify the operation of the monitoring camera system shown in 
FIG. 1 and is an example of the overlap and swivel applied to the angle of view of the 
moving body detection camera. 
25 FIG. 11 is a flowchart showing the constitution of a monitoring camera system 

according to a working example of the present invention. 

Key to referencing numerals 



11, 111 mobile body detection cameras 

12, 112 attitude control means 

30 21, 121 monitoring cameras 

23 image output device 

31, 32 camera angle of view storage means 

40, 140 image processing units 

41 moving body extraction means 

35 42 feature vector extraction means [9] 

43 feature and position information storage means 

44 target body tracking means 

45 camera angle of view specifying means 

112, 122 universal heads 

40 123 image splitter 

124 display 

125 VTR device 

131, 132 preset controllers 
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FIG. 3 
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TRANSLATOR'S NOTES 



General Note: The 3apanese specification of which the above is the translation is 
noticeably lacking in clarity at several key points, mainly concerning (i) the interrelation 
between the tracking nnechanisms of the two canneras and (ii) the question of the number 
(singular/plural) of various components/operations. My translation of some of the resulting 
ambiguous Japanese is therefore based, at several points, on my understanding of the 
substance of the invention. 

1. "Cailing" here is evidently used in a computer program sense. However, I wonder if 
"retrieving" would not have been more appropriate. 

2. Sic. The writer perhaps uses the term "decision camera" to indicate that it is this camera 
which provides images that allow a decision to be made regarding the identity of a moving 
body. 

3. The Japanese which I have translated as "in such manner that no part of the monitoring 
area is left uncovered" is literally, "so that no gaps will form within the monitoring area". 

4. Japanese does not generally distinguish between singular and plural, and it Is therefore 
difficult to interpret some of the statements made in this paragraph. At the beginning of 
the paragraph the writer seems to be discussing the processing applied to a single moving 
body ("When a moving body enters the image capture area..."), whereas in the 
subsequent statements "obtains the features and position for each moving body region" 
and ''compares the information relating to each moving body", "each" seems to refer to 
each of various moving bodies that may be detected at one time. However, despite the 
presence of more than one moving body being detected, target body tracking means 44 is 
able to identify one of these moving bodies as the target of interest to be tracked. (In this 
connection, see paragraph 30, where it is stated that data relating to moving bodies that 
are not of interest, such as the swaying of tree branches, are deleted). 

5. Although the Japanese in this long sentence does not indicate in any way whether "camera 
angle of view", "image capture area" and "table number" is to be interpreted as singular or 
plural in number, I. have assumed that each of these can be either singular or plural. In 
this invention both the moving body detection camera and the monitoring camera (the 
later employed for close-up tracking) are attitude-controllable, and it would seem that the 
angles of view employed are independently adjustable. However, there may also be times 
in the course of the tracking process when only one angle of view has to be altered, or 
indeed when neither angle of view has to be altered. 

6. The Japanese which I have translated as "for an example where the image capture area. of 
the moving body detection camera has been divided into 5g5 blocks" is literally "for an 
example where it has been divided into 5*5 blocks". My expansion of "it" is aimed at 
clarifying the writer's presumable meaning. 

7. Sic. 

8. The Japanese text erroneously has "attitude control means 31 and 32" here. I have made 
the necessary correction. 

9. Sic. Note that everywhere else in the Japanese text, 42 has been identified as the "feature 
and position extraction means", rather than the "feature vector extraction means". 
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